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(57) Abstract 

A rate-distortion optimized embedding (RDE) coder (Fig. 5) optimizes rate-distortion performance by coding information bits in the 
order of their R-D slope. RDE allocates the available coding bits first to the information bit with the steepest R-D slope (fig. 9), which 
indicates the largest distortion decrease per coding bit. The resultant RDE bitstream can be truncated at any point and still maintain an 
optimal R-D performance. To avoid the overhead of coding order transmission, an expected R-D slope is calculated by both the encoder and 
the decoder from previous RDE coded bits. A probability estimation table from a QM arithmetic coder (fig. 5, code 40) allows the R-D slope 
to be derived using a lookup table operation. The rate-distortion optimized embedding (RDE) coder significantly improves the coding 
efficiency over a wide range of bit rates. 
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AN EMBEDDED IMAGE CODER 
WITH RATE-DISTORTION OPTIMIZATION 

BACKGROUND OF THE INVENTION 

5 This invention relates to embedded coding techniques and more particularly to an 

embedded image coder with improved rate-distortion. 

Embedded image coding improves coding performance and also allows a bitstream 
to be truncated at any point and still decode a reasonably good image. Some 
representative embedded coding techniques include the embedded zerotree wavelet coding 

10 (EZW) discussed in J. Shapiro, "Embedded image coding using zerotree of wavelet 

coefficients", IEEE Trans. On Signal Processing, vol. 41, pp.3445-3462, Dec. 1993, the set 
partitioning in hierarchical trees (SPIHT) discussed in A. Said, and W. Pearlman, "A new, 
fast and efficient image codec based on set partitioning in hierarchical trees", IEEE Trans. 
On Circuit and System for Video Technology, Vol. 6, No. 3, Jun. 1996, pp. 243-250 and 

15 the layered zero coding (LZC) discussed in, D Taubman and A. Zakhor, "Multirate 3-D 
subband coding of video", IEEE Trans. On Image Processing, Vol. 3, No. 5, Sept. 1994, 
pp.572-588. 

The ability to adjust the compression ratio by simply truncating the coding bit- 
stream makes embedding attractive for a number of applications such as progressive image 

20 transmission, internet browsing, scalable image and video databases, digital cameras, low 
delay image communications, etc. Taking internet image browsing as an example, em- 
bedded coding requires storage on a server of only one copy of a high quality image. De- 
pending on user demand, channel bandwidth conditions, and browser monitor quality, 
selectable amounts of the image bit stream can be delivered to the browser. At an early 

25 stage of browsing, images can be retrieved with coarse quality so that a user can quickly 
go through a large number of images before choosing one image of interest. The chosen 
image can then be fully downloaded with a better quality level. During the download pro- 
cess, the image quality is gradually refined. The user can terminate the download process 
as soon as the quality is satisfactory to discern the image. - - 

30 Embedded coding allows the bitstream to be arbitrarily truncated. However, exist- 

ing embedded coding techniques are not optimized at every truncation point in the bit 
stream. Thus, if the encoded bitstream is truncated at random points in the bit stream, the 
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image produced by the bit stream up to the point of truncation does not necessarily pro- 
duce optimal image quality. 

It is known that a fixed rate coder achieves optimality if the rate-distortion (R-D) 
slopes of all coded coefficients are the same, T. M. Cover and J. A. Thomas, "Elements of 
5 information theory 11 , Chapter 13, John Wiley & Sons Inc, 1991. The criterion was used in 
rate control to adjust the quantization step size Q of each macroblocks, in which case the 
coding of video was optimal when the R-D slopes of all macroblocks were constant. See 
L.-J. Lin, A. Ortega and C.-C. J. Kuo, "Rate control using spline-interpolated R-D charac- 
teristics", SPIE: Visual Communication and Image Processing, vol. 2727, Orlando, FL, 

10 Apr. 1996, pp. 1 1 1-122 and K. Ramchandran, A. Ortega and M. Vetterli, "Bit allocation 
for dependent quantization with applications to multiresolution and MPEG video coders", 
IEEE Trans. On Image Processing, Vol. 3, No. 5, Sep. 1994, pp. 533-545. 

Xiong and Ramchandran use the constant rate-distortion slope criterion to derive 
the optimal quantization for wavelet packet coding. Z. Xiong and K. Ramchandran, 

15 "Wavelet packet-based image coding using joint space-frequency quantization", First 
IEEE International Conference on Image Processing, Austin, Texas, Nov. 13-16, 1994. 
However, Xiong and Ramchandran do not optimize rate-distortion optimization for em- 
bedded coders. 

The R-D slopes of significance identification and refinement coding are different, 
20 and by placing the significance identification before the refinement coding, the coding 
efficiency can be improved. However, the improvement is limited since only the coding 
order of a few coefficients is effected. See Li, Cheng and Kuo J. Li, P. Cheng and C. -C. 
J. Kuo, "On the improvements of embedded zerotree wavelet (EZW) coding", SPIE: Vis- 
ual Communication and Image Processing, vol. 2601, Taipei, Taiwan, May. 1995, pp. 
25 1490-1501. 

Thus, a need remains for an embedded coder that optimizes rate-distortion at many 
different truncation points in an encoded bit stream. 

SUMMARY OF THE INVENTION 
A rate-distortion optimised embedding (RDE) coder optimises rate-distortion 
30 performance by coding coefficients in order of their R-D slope. RDE allocates the 

available coding bits first to the coefficient with the steepest R-D slope, which indicates 
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the largest distortion decrease per coding bit. The resultant RDE bitstream can be 
truncated at any point and still maintain an optimal R-D performance. To avoid the 
overhead of coding order transmission, an expected R-D slope is calculated by both the 
encoder and the decoder from pervious RDE coded bits. A probability estimation table 

5 from a QM arithmetic coder allows the R-D slope to be derived using a lookup table 
operation. The RDE rate-distortion optimisation significantly improves the coding 
efficiency over a wide bit rate range. 

The RDE embedded coding method digitizes the image and then transforms the 
image to form a set of coefficients each comprising multiple bits. The RDE coder 

10 repeatedly analyzes all the candidate bits before encoding individual bits. The candidate 
bits comprise one most significant unencoded bit from each coefficient. Individual rate- 
distortion values for the candidate bits represent a ratio of information content in the bit 
per cost in transmitting the bit. Selected bits are then encoded according to the associated 
distortion reduction values for the bits. 

15 In one embodiment, one bit among the candidate bits having a maximum distortion 

reduction value is encoded. The candidate bits are updated comprising a next less 
significant bit in the encoded bit coefficient and the remaining unencoded candidate bits. 
The bit among the candidate bits with the highest distortion reduction value per coding bit 
is encoded next. The process is repeated until the RDE coder reaches a specified bit rate. 

20 In another embodiment, a threshold value is set and the candidate bits having 

distortion reduction values greater than the threshold are encoded in each scan. The 
candidate bits are updated comprising a next less significant bit in the encoded bit 
coefficient and the remaining unencoded candidate bits. The threshold value is then 
reduced and a next set of unencoded candidate bits are processed. The candidate bits in 

25 the next set having distortion reduction values greater than the reduced threshold value are 
then encoded. The process is repeated until the RDE coder reaches the predetermined bit 
rate. 

The foregoing and other objects, features and advantages of the invention will 
become more readily apparent from the following detailed description of a preferred 
30 embodiment of the invention, which proceeds with reference to the accompanying 
drawings. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is graph comparing a conventional encoded bit stream with an encoded bit .. 
with rate-distortion optimization according to the invention. 

FIG. 2 is a table showing a bit array after transformation. 
5 FIG. 3a is a table showing a coding order of a prior art conventional coder. 

FIG. 3b is a table showing a coding order of a prior art embedded coder. 

FIG. 3 c is a table showing a coding order for a rate-distortion optimized embedded 
coder (RDE) according to the invention. 

FIG. 4 is a table showing RDE estimation of the rate-distortion slopes based on 
10 previously encoded bits. 

FIG. 5 is a block diagram of a RDE coder according to the invention. 

FIG. 6 is a table showing significance identification bits, refinement bits and sign 

bits. 

FIG. 7 shows an image context for a prior art QM-coder. 
15 FIG. 8 is a graph illustrating coding interval subdivisions used for predicting rate- 

distortion slopes in the RDE coder. 

FIG. 9 is a graph showing a rate-distortion slope modification factor for 
significance identification. 

FIG. 10 is a flow chart showing how the RDE coder in FIG. 5 performs rate- 
20 distortion optimized embedded coding according to the invention. 

FIG. 1 1 is a graph comparing RDE rate-distortion with other encoding techniques. 

FIG. 12 is table showing another example of a coding order for the RDE coder. 

FIG. 13 is a table comparing rate-distortion for the different encoding techniques in 
FIG. 11. 

25 DETAILED DESCRIPTION 

To achieve optimal rate-distortion performance at every truncation point in a bit 
stream, symbols are encoded in order of their steepest rate-distortion (R-D) slope. The 
result of the RDE encoder is illustrated in FIG. 1 . There are five symbols a, b, c ? d and e 
that are be coded independently. The coding of each symbol requires a certain amount of 
30 bits represented by horizontal rate axis R. Encoding bits result in a certain amount of 
distortion decrease represented by vertical distortion axis D. Conventional sequential 
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coding in the order of symbol a to e gives the R-D curve shown as solid line 12. The R-D 
curve shown as dashed line 14 shows the effects when coding is reordered so that the 
symbol with the steepest R-D slope is encoded first. Though both performance curves 
reach the same final R-D point, the coder generating the dashed line 14 performs much 
5 better when the output bitstream is truncated at any intermediate bit rate. 

The rate-distortion optimized coder (RDE) according to the invention, allocates the 
available coding bits first to the coefficient with the steepest R-D slope. The steepest R-D 
slope is defined as the coefficient bit that provides the largest distortion decrease in the 
image per coding bit. When all symbols have been coded, the coding is lossless and RDE 

10 may generate similar performance as non-RDE coding techniques. However, RDE 

optimization outperforms conventional embedded coding at intermediate bit rates. If the 
filter or transform used to encode image data is not integer based, RDE always 
outperforms convention embedding since lossless coding can not be achieved. 

The two primarily steps performed in RDE are R-D slope calculation and coeffi- 

15 cient selection. To avoid sending the overhead of coding order, RDE can be based on an 
expected R-D slope that is calculated independently by both an encoder and a decoder. 
The expected R-D slope can be calculated using a lookup table operation in conjunction 
with a probability estimation table of a QM-coder. Operation of QM-coders are described 
in W. Pennebaker and J. Mitchell, IBM, "Probability adaptation for arithmetic coders", 

20 U.S. Patent No. 5,099,440, Mar. 24, 1992; and D. Duttweiler and C. Chamzas, "Probabil- 
ity estimation in arithmetic and adaptive Huffman entropy coders", IEEE Trans. On Image 
Processing, vol. 4 no. 3, Mar. 1995, pp. 237-246, which are both herein incorporated by 
reference. 

Implementation of Rate-Distortion Optimized Embedding (RDE) 

25 In the discussion below, it is assumed that an image has already been converted 

into the transform domain. Any transform can be used with the embedded coding tech- 
nique including a wavelet decomposition or DCT transform. For simplicity, the RDE is 
described in terms of a wavelet decomposition. The index of a transform coefficient is 
denoted by i=(s,d,x,y), where s is a scale of the wavelet decomposition, d is a subband of 

30 decomposition which includes LL, LH, HL and HH, and x, y are spatial positions within 
the subband. 
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The first and second letter in d represent the filter applied in the vertical and hori- 
zontal direction, respectively. The letter L represents a low-pass filter, and H represents a 
high-pass filter. N denotes the total number of transform coefficients. The coefficient at 
index position i is denoted by wj. Assume the coefficients have already been normalized 

5 through the division of the maximum absolute value of the transform coefficients Tq: 

w i 

w'j— with T 0 = max |wj| (1) 
0 i 

To simplify the notation, the apostrophe in w\ is eliminated and the normalized 
transform coefficients are simply denoted as w^. Because wj is between -1 and +1, it can 
be represented by a stream of binary bits as: 
10 ±0.b 1 b 2 b 3 --bj-- (2) 

where bj is the j-th most significant bit, or alternatively referred to as the j-th coding layer 
of coefficient Wj. For simplicity, the rate-distortion optimized embedding (RDE) discus- 
sion below, the coding symbol is defined as the smallest unit sent to the entropy coder and 
is either one single bit bj of the coefficient Wj or the sign of Wj. However, the scope of the 

15 invention can easily extended to encoding arrangements, where the coding symbol may 

consist of encoding a group of bits from each coefficient at the same time, as in the case of 
the embedded zerotree wavelet coding (EZW) or the set partitioning in hierarchical trees 
(SPIHT). 

A sample bit array produced by a ID-wavelet transform is shown in FIG. 2. The i- 
20 th row of the bit array represents the transform coefficient wj, and the j-th column of the 
bit array represents the bit plane bj. The most significant bit is located at the left most 

column and the least significant bit is located at the right most column. 

The encoding order of the bit-array is different for conventional, embedded, and 
rate-distortion optimized embedded coders. A conventional coder such as a JPEG coder or 
25 a MPEG coder first determines the quantization precision, or equivalently, the number of 
bits to encode each coefficient, then sequentially encodes one coefficient after another 
with certain entropy coding. Using the bit array of FIG. 2 as an example, the conven- 
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tional coding is ordered row by row as further shown in FIG. 3a. A first row 16 is en- 
coded, then a second row 1 8 is encoded and so on until the entire bit plane is encoded. 

Embedded coding is different from the conventional coding shown in FIG. 3 a be- 
cause the image is coded bit-plane by bit-plane (column by column) as shown in FIG. 3b. 
5 The first bit of each coefficient in a first column 20 is encoded starting from the first bit in 
coefficient w 0 and ending with the first bit in coefficient w 7 . The embedded coder then 
starts encoding a second column 22 of coefficients starting from the second bit of coeffi- 
cient w 0 and ending with the second bit of coefficient w 7 . The embedded bitstream can be 
truncated and still maintain some image quality since the most significant part of each 
10 coefficient is coded first. It is also suited for progressive image transmission because the 
quality of the decoded image gradually improves as more and more bits are received. On 
the other hand, the coding order of the embedded coding technique shown in FIG. 3b is 
not optimized for progressive image transmission. 

RDE calculates the R-D slope A, ; for each bit bj and encodes the bit with the largest R-D 

15 slope. The actual coding order of RDE depends on the calculated R-D slope and is image depend- 
ent. An example of the coding order of RDE is shown in FIG. 3c where a first group of bits 24 
shown in cross-hatching comprising the first bits of coefficients w 0 through w 4 are encoded first 
and then a second group of bits 26 shown in opposite cross-hatching are encoded next. The order 
of encoding in the second group of bits 26 begins by encoding the second bit of coefficient w 0 , 

20 then the second bit of coefficients w 2 and then coefficient w 3 . Finally the first bits of coefficients 
w 4 through w 7 are encoded. The same process is conducted for the checkered bits in group 28 
starting with the second bit of coefficient w l and ending with the second bit of coefficient w 7 . 
The process is continued until a specified bit rate is reached or all coefficients are encoded. An- 
other more elaborated coding order of RDE is shown in table 1 of FIG. 12. The order of coding, 

25 the symbol to encode and its value are listed in column 1, 2 and 3, respectively. 
Expected Rate-Distortion Slope 

If the optimization is based on the actual rate-distortion (R-D) slope, the decoder 
has to be informed of the order of coding. A large bit overhead is required to transmit the " 
location of the symbol with the largest actual R-D slope and can easily nullify any advan- 

30 tages that can be brought up by rate-distortion optimization. To avoid transmitting the 
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coding order, a prediction technique is used to calculate an estimated R-D slope in both the 
encoder and the decoder. 

FIG. 4 shows how the rate-distprtion slope is estimated based on the transmitted 
bits. The bits marked by horizontal hatching have already been coded. The bits marked 
5 with a checkered pattern are the next candidate bits for encoding. The letter S indicates 
the R-D slope for the bit will be estimated using a significance identification mode. The 
letter R indicates the R-D slope for the bit will be estimated using a refinement estimation 
mode. 

Suppose at a certain coding instance, the most significant (nj-1) bits of coefficient 
10 Wj have been encoded. The next set of candidate bits under consideration is the nj-th bit of 

Wj, i=l,' * *,N. RDE calculates the expected R-D slope X t for each candidate bit b n ., and 

encodes the one bit with the largest X i value. The expected R-D slope X i is based on the 
coding layer nj, the significance status of coefficient Wj (whether all of the previous (n^-l) 
bits of w^ are zero), and the significance status of its surrounding coefficients. 
15 RDE estimates the distortion decrease per coding bit if bit b n . is coded. Since the 

information used to calculate the expected R-D slope can be derived independently by the 
decoder from the previously transmitted bits, the decoder can follow the coding order of 
the encoder without any overhead transmission. RDE encodes the symbol that gives the 
maximum expected distortion decrease per bit spent, thus achieving optimized R-D em- 

20 bedded coding performance as shown by the dashed line 14 in FIG. 1 . 

An encoder 30 utilizing the RDE coder is shown in FIG. 5. Digitized images 31 
are converted into coefficients by a transform 32. The RDE coder comprises a R-D slope 
calculator 34 that estimates the R-D slopes for the coefficient bits and a symbol selector 36 
that selects bits for subsequent encoding according to the calculated R-D slope values. A 

25 quantizer 38 quantizes the RDE ordered bits and an arithmetic coder 40 further encodes 

the quantized bits. The output of quantizer 38 is inverse quantized and subtracted from the 
coefficient values output from transform 32. 

Compared with traditional embedded coders, there are two key operations in RDE, 
R-D slope calculation and coefficient selection. Both operations have to be efficient so that 

30 the computational complexity of RDE remains low. 
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Calculation of the Rate-Distortion Slope 

The coding technique described below allows the expected R-D slope to be derived 
using a lookup table operation. In RDE, the coding of candidate bits b n . is encoded using 

5 either a significance identification mode or a refinement mode. If all previously coded bits 
in coefficient wj are Os, bj=0 for j=l— nj-1, the significance identification mode is used to 
encode bit b n ., otherwise, the refinement mode is used. For convenience, coefficient wj is 

called insignificant if all its previously coded bits are Os. The insignificant coefficient is 

reconstructed with value 0 at the decoder side. When the first nonzero bit b„ is encoun- 

n i 

10 tered, coefficient Wj becomes significant. The sign for the coefficient needs to be encoded 

to distinguish the coefficient value between positive and negative, and it becomes non-zero 
at the decoder. From that point on, the refinement mode is used to encode the remaining 
bits of coefficient wj. 

Referring to FIG. 6, bits encoded during the significant identification mode are 
15 marked with horizontal hatching and bits encoded during the refinement mode are marked 
with dots. The bits marked by checkerboard patterns are sign bits, which are encoded 
when a coefficient just becomes significant. The expected R-D slopes and coding methods 
for significance identification and refinement are different. 

In significance identification, the coded bit is highly biased toward '0\ i.e., non- 
20 significance. The result of significance identification is encoded with a QM-coder, which 
estimates the probability of significance of coefficient Wj (denoted as pj) with a state 

machine, and then arithmetic encodes it. As shown in FIG. 7, a context of 7 bits is used, 
which consists of 6 bits 44 shown in light shading representing the significant status for 6 
spatial neighbor coefficients and 1 bit 44 shown in dark shading representing the 
25 significant status of the coefficient which corresponds to the same spatial position but one 
scale up (i.e., in the lower resolution band of) the current coefficient Wj. By monitoring - _ 

the pattern of past Os ('insignificance') and Is ('significance') under the same context (i.e., 
the same neighborhood configuration), the QM-coder estimates the probability of signifi- 
cance pj of the current symbol being analyzed. To explain further, if there were n 0 0 sym- 
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bols and nj 1 symbols in the past coding with the same context, the probability p that the 
current symbol appears 1 can be calculated by Bayesian estimation as: 

m + 5 

P= ' 71 ( 3 ) 
where 5 is a parameter between [0,1] which relates to the a priori probability of the coded 
5 symbol. The probability p is associated with a state. Depending on whether the coded 
symbol is 1 or 0, it increases or decreases the probability p and thus transfers the coder to 
another state. By merging of the state of similar probabilities and balancing between the 
accuracy of probability estimation and quick response to the change in source characteris- 
tics, a QM-coder state table can be designed. 

10 The QM-coder and its probability estimation, are described in W. Pennebaker and 

J. Mitchell, IBM, "Probability adaptation for arithmetic coders", US patent 5,099,440, 
Mar. 24, 1992; D. Duttweiler and C. Chamzas, "Probability estimation in arithmetic and 
adaptive Huffman entropy coders", IEEE Trans. On Image Processing, vol. 4 no. 3, Mar. 
1995, pp. 237-246; and W. B. Pennebaker and J. L. Mitchell, "JPEG still image data com- 

15 pression standard", New York: Van Nostrand Reinhold, 1993. 

In general, the probability estimation is a table transition operation. The estimated 
probability of significance pj is used not only for arithmetic coding, but also for the calcu- 
lation of the R-D slope X t . On the other hand, the refinement and sign bits are equilibrium 
between '0' and T. An arithmetic coder encodes them with fixed probability 0.5. 
20 RDE calculates the expected R-D slope X ( for all the candidate bits b n ., which is the 

average distortion decrease divided by the average coding rate increase: 

E[ADj] 

The expected R-D slope can not be calculated by averaging the distortion decrease 
per coding rate: 

10 
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10 



15 



ADj 



(5) 



It is just like the calculation of the average speed of a vehicle traveling through different 
segments with varying speed. Its average speed is equal to the total travel distance divided 
by the total travel time, not the average of speed of different segments. 

Referring to FIG. 8, suppose before coding bit b n ., coefficient Wj is known to be 

within interval (Mq ^, Mj ^) with decoding reconstruction r^. Coding of bit b n . supplies 

additional information of coefficient wj and restricts it into one of K subintervals 

(^k a ? a) with decoding reconstruction r^ a , k=0 ? - • -,K-1. The interval boundaries 

satisfy the relationship: 

M 0,b= M 0,a < M l,a<- • - <M K,a= M l,b ( 6 ) 
Whereas the decoding reconstruction is usually at the center of the interval: 

rb=(M 0 ,b+M lsb )/2, (7) 

and r k>a -(M k a +M k+1 ?a )/2, k=0,- ■ -,K-1 . (8) 
The average distortion decrease and the average coding rate increase are calculated as: 

¥ M .T[(x-r b ) 2 - (x-r k a ) 2 ]p(x)dx , (9) 
k=0 M k , 

and E[ARi]= 2,- p k lo g2 P k with P k = J P(x)dx , (10) 

k=0 M k.a 

where p(x) is the a priori probability distribution of the coding symbol which is normal- 
ized so that the probability of the entire interval (Mq ^, M^) is equal to 1: 

Jp(x)dx=l (11) 

M 0 , b 

20 In the case the candidate bit b n . undergoes significance identification, coefficient 

Wj is insignificant within interval (-2T n .,2T n .) before the coding of b n ., where T n ^=2~ n i is 
the quantization step size determined by the coding layer n^. After the coding of b n ., coef- 
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ficient wj may be negatively significant with interval (-2T n .,-T n .], positively significant 

with interval [T n .,2T n .), or still insignificant with interval (-T n .,T n ). Thus there are three 

possible segments after significance identification with segment boundaries: 

M 0,b=M 0 ,a = -2T n ., M l5a = -T n ., M 2 a = T n ., and M 3>a = M 1>b = 2T n ., (12) 

5 The decoding reconstruction value before significance identification is: 

r D =0 (13) 
The decoding reconstruction values of each segment after significance identification are: 
r o. a =~ l -5T ni , r Ui =0, r 2r3 =\.5T ni , respectively. (14) 
Because the probability of significance pj, i.e. the probability b n .=l which is esti- 

10 mated by the QM-coder, can be formulated as: 

-T 2T 
n i n i 

p r / p(x)dx+ / p(x)dx. (15) 
"2T ni T n . 

Assuming that the a priori probability distribution within the significance interval is uni- 
form, p(x) is formulated as: 

p(x)=~F-, for T n <|x|<2T n . (16) 

^ 1 n j 1 1 

15 By substituting equations (12), (13), (14) and (16) into (9) and (10), the av- 

erage distortion decrease and average coding rate increase are calculated for significance 
identification as: 

EtAD^i 2.25T n . 2 (17) 

E[AR i ]=(l-p i )[-log 2 (l-p i )]+2 y(-log 2 y) ^^(pi) (18) 

20 where H(p) is the entropy of a binary symbol with probability of 1 equal to p: 

H(p) - -p log 2 p - (1-p) log 2 (l-p) (19) 
Note that the average distortion (17) is not related to the a priori probability within^ ~ 
insignificance interval (-T n .,T n .), because within that interval the decoding values before 
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and after coding are both 0. The expected R-D slope for significant identification is de- 
rived from (17) and (18) as: 

g[ADj] 2 ' 25T n i 2 2 
*<i,sig - EtAR^^H-HCpiVpi ~ f s (p i ) T nj ( 20 ) 

Function f g (p) is the significance R-D slope modification factor defined as: 

5 fs(p) = ~m > (21) 

p 

FIG. 9 plots the R-D slope modification factor for significance identification. The symbol 
with higher probability of significance has a larger R-D slope and is thus favored to be 
coded first. The calculation of the R-D slope is only based on the coding layer nj and the 
probability of significance pj, which is in turn estimated through the QM-coder state. 

10 The expected R-D slope for refinement coding is similarly derived, where coeffi- 

cient wj is refined from interval [Sj,Sj+2T n .) to one of the two segments [Sj,Sj+T n .) or [Sj 

+T nj' S i +2T nj)- T nj ==2 " ni is a 8 ain the quantization step size determined by the coding layer 
nj, and Sj is the start of the refinement interval which is determined by the previously 
coded bits of coefficient w^ The segment boundaries are: 

15 M 0,b= M 0,a = S i> M l,a = S i +T n r and M 2,a = M l,b =s i+ 2T n r ( 22 ) 

and the corresponding decoding reconstruction values are: 

r b =S i +T n r r 0,a= S i +0 - 5 T nr and r 1?a = S i+ L5T n .. (23) 
Assuming that the a priori probability distribution within interval [Sj,Sj+2T n .) is 
uniform, we have: 

20 P(x)=2f-> for Sj<x< Si+2T n . , (24) 

n i 1 

The average distortion decrease and coding rate increase for refinement coding are calcu- 
lated as: 

E[ADj] =0.25 T n . 2 (25) 
E[ARj] =1 (26) 
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The expected R-D slope for refinement coding is thus: 

E[ADj] 

\rerE[ARi\^ 0 ' 25 V (27) 
Comparing (20) and (27), it is apparent that for the same coding layer nj, the R-D 

slope of refinement coding is smaller than that of significance identification whenever the 
5 significance probability pj is above 0.01. Thus in general the significance identification 
coding should be conducted before the refinement coding. 

The a priori probability distribution of coefficient wj can also be modeled with a 

Laplacian distribution. In this case, the R-D slope for significance identification and re- 
finement becomes: 

10 ^i.sig = f s(Pi) gsig(s.T n .) T n . 2 (28) 

Vef =0 - 25 gref< s ' T n i ) ^ (29) 
where <r is the variance of Laplacian distribution that can also be estimated from the al- 
ready coded coefficients, g s j g (a,T) and g ref (a,T) are Laplacian modification factors in the 



form: 



1 



15 g sig( a ' T ) = 235 



3a 3e 



,-T/o- 



0.75 + - 



T " Le-T/o 



V - i-e 

f 2^e-T/o^( i +e -2T/cr^ 
0.75 



(30) 



g ref (a,T)=4^.,. , ^ m j 

However, experiments show that the additional performance improvement pro- 
vided by the Laplacian probability model is minor. Since the uniform probability model is 
much simpler to implement, it is used throughout the experiments described below. 
20 Because the probability of significance pj is discretely determined by the QM- 

coder state, and the quantization step size T n . associated with the coding layer nj is also 

discrete, both the R-D slope of significance identification (20) and refinement (27) have a_ „ 
discrete number of states. For fast calculation, (20) and (27) are pre-computed and stored 
in a table indexed by the coding layer n^ and the QM-coder state. Computation of the R-D 
25 slope is thus a lookup table operation. The R-D slope of refinement needs one entry per 
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coding layer. The R-D slope of significance identification needs two entries per coding 
layer and per QM-coder state, as each QM-coder state may correspond to the probability 
of significance pj (if the most probable symbol is 1) or the probability of insignificance 1- 
Pj (if the most probable symbol is 0). Therefore, the total number of entries M in the 
5 lookup table is: 



where K is the maximum coding layer, L is the number of states in the QM-coder. In the 
current implementation, there are a total of 1 13 states in the QM-coder, and a maximum of 
20 coding layers. This brings up a lookup table of size 4540 entries. 

10 Coefficient Selection 

The second RDE step performed by symbol selector 36 in FIG. 5 is selecting the 
coefficient with the maximum expected R-D slope. This may be done by exhaustive 
searching or sorting over all candidate bits, which is computationally expensive. An alter- 
native implementation uses a threshold-based approach. The concept is to setup a series of 

15 decreasing R-D slope thresholds y 0 > y , >• ■ ■> y n >• • and to scan the whole image re- 
peatedly. The symbols with R-D slope between y n and y n+l are encoded at iteration n. 
The threshold based rate-distortion optimization sacrifices a little bit performance as sym- 
bols with R-D slope between y n and y B+1 can not be distinguished. However, the coding 
speed is much faster as the search for the maximum R-D slope is avoided. 

20 The entire coding operation of RDE is shown in FIG. 10. The left half of FIG. 10 

shows the main RDE operation flow and the right half is a detailed description of R-D 
slope calculations and symbol coding. Since the symbols of significance identification and 
refinement are treated differently, they are depicted with separate branches in R-D slope 
calculation and symbol coding. 

25 1) Initialization 

The image is decomposed by a transform, such as the wavelet transform, in step 52. An 
initial R-D slope threshold y is set to y 0 in block 54, with: 



M=2KL+K 



(32) 
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2) Scanning 

The entire image is scanned in step 56 top-down from the coarsest scale to the finest 
scale. Scale is defined as resolution of the wavelet decomposition. Within each scale, 
subbands are coded sequentially with order LL, LH, HL and HH. The coder follows the 
raster line order within the subband. 

3) Calculation of the expected R-D slope 

The expected R-D slope is calculated for the candidate bit of each coefficient in blocks 
66 and 68. Decision block 64 determines whether the coefficient is significant (whether a 
binary 1 has yet occurred in the string of coefficient bits). The expected R-D slope is 
calculated according to equation (20) in block 66 for the significance mode or equation 
(27) in block 68 for the refinement mode. The calculation of the R-D slope is typically a 
lookup table operation indexed by the QM-coder state and the coding layer nj. 

4) Coding decision 

The calculated R-D slope is compared with the current threshold y in decision block 70. 
If the R-D slope is smaller than y for the current iteration, the RDE coder processes the 
next coefficient by jumping back to decision block 64. Only the candidate bits with R-D 
slope greater than y are encoded. 

5) Coding of the candidate bit 

Depending again on whether the coefficient is already significant, the candidate bit 
is coded with significance identification in block 74 or with refinement in block 76. The 
QM-coder with context designated in FIG. 7 is used for significance identification coding 
in block 74. A fixed probability arithmetic coder is used to encode the sign and refinement 
in block 76. The sign bit is encoded right after the coefficient becomes significant. As 
mentioned above, the QM-coding and arithmetic coding are well known to those skilled in 
the art and are therefore not described in further detail. 

6) Iteration " ~ 

After the entire image has been scanned, the R-D slope threshold is reduced in block 66 
by a factor of a : 
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Y (34) 

In the current implementation, a is set to 1.25. The RDE coder checks if the assigned 
coding rate is reached in decision block 62. For example, the RDE coder determines if the 
compressed image has reached a predetermined number of bits. If the coding rate is not 
5 reached, the RDE coder jumps back to block 56 and encoding repeated for the remaining 
unencoded bits at the new lower threshold value. 

Experimental Results 

Extensive experiments were performed to compare RDE with other existing algo- 
rithms. Test images were identified as Lena, Boats, Gold and Barbara. The image Lena is 

10 of size 512x512, all other images are of size 720x576. The test image is decomposed by a 
5-level 9-7 tap biorthogonal Daubechies filter. It is then compressed by the layered zero 
coding LZC, proposed by Taubman and Zakhor, the set partitioning in hierarchical trees, 
as SPIHT proposed by Said and Pearlman, and the rate-distortion optimized embedding 
(RDE), respectively. The SPIHT coder is used as a reference coder. RDE essentially shuf- 

15 fles the bitstream of LZC and improves its embedding performance. Therefore RDE is 
compared with LZC to show the advantage of rate-distortion optimization. The initial 
probability of QM-coder in RDE is set at equilibrium, (i.e., the probabilities of 1 of all 
contexts are equal to 0.5). No pre- statistics of image is used. The compression ratio in the 
experiment is chosen to be 8:1 (1.0 bits per pixel (bpp)), 16:1 (0.5bpp), 32:1 (0.25bpp) and 

20 64:1 (0.125bpp). Since all three coders are embedded coders, the coding can be stopped at 
the exact bit rate. 

The comparison results are shown in Table 2 of FIG. 13. The coding rate in bits per 
pixel is shown in column 2, the peak signal-to-noise ratio (PSNR) of LZC and SPIHT are 
shown in columns 3 and 4, and the PSNR of RDE and its gain over LZC and SPIHT are 

25 shown in column 5, 6 and 7, respectively. The R-D performance curve of the Barbara im- 
age is plotted in FIG. 11. The R-D curve of RDE is bold line 78, LZC is represented by 
solid line 80 and SPIHT is represented by dotted line 82. The R-D curve in FIG. 1 1 calcu- 
lates one PSNR point every increment of a few bytes and indicates that RDE outperforms 
both LZC and SPIHT. The performance gain of RDE over LZC ranges from 0.1 to 0.8dB, 

30 with an average of 0.3dB. The gain shows the performance advantage of rate-distortion 
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optimization. From FIG. 1 1, it can be observed that the R-D performance curve of RDE is 
also much smoother than that of LZC. The effect is a direct result of rate-distortion opti- 
mization. With the embedded bitstream organized by decreasing rate-distortion slope, the 
slope of the resultant performance curve decreases gradually, resulting in the smooth 
looking R-D curve of RDE. The performance gain of RDE over SPIHT ranges from -0,1 
to l.OdB, with an average of 0.4dB. 

Thus, rate-distortion optimized embedded coder (RDE) improves the performance 
of embedded coding at every possible truncation point by coding first the symbol with the 
steepest R-D slope. That is, at each coding instance, RDE spends bits encoding the coding 
symbol with the largest distortion decrease per coding bit. For synchronisation between 
the encoder and the decoder, RDE uses the expected R-D slope, which can be calculated 
independently by both the encoder and the decoder. It also takes advantage of the prob- 
ability estimation table of the QM-coder so that the calculation of the R-D slope can be 
performed using a lookup table operation. 

Having described and illustrated the principles of the invention in a preferred 
embodiment thereof, it should be apparent that the invention can be modified in 
arrangement and detail without departing from such principles. I claim all modifications 
and variation coming within the spirit and scope of the following claims. 
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CLAIMS 

1 . An rate-distortion optimization method for embedded coding of an image, 
5 comprising: 

digitizing the image; 

transforming the image to form a set of coefficients; 
quantizing the coefficient to form a set of symbols; 
calculating the distortion reduction values for each symbol representing 
10 a ratio of information content in the symbol per cost in transmitting the symbol; and 

encoding the symbols according to a descending order of distortion reduction 

value. 

2. A rate-distortion optimization method for embedded coding of an image, 
15 comprising: 

digitizing the image; 

transforming the image to form a set of coefficients each comprising multiple bits; 

repeatedly analyzing a group of candidate bits before encoding individual bits, the 
candidate bits comprising the most significant unencoded bit from each coefficient; 
20 determining distortion-reduction values for the bits among the candidate bits 

representing a ratio of information content in the bit per cost in transmitting the bit; 

encoding a selected bit among the candidate bits with an entropy coder; and 

substituting the coded bit in the candidate bits with the next most significant 
unencoded bit of the coefficient. 

25 

3. A method according to claim 2 wherein encoding selected bits comprises 
encoding one bit among the candidate bits having a maximum distortion reduction value 
per coding bit spent. 

30 4. A method according to claim 2 including the following: 

setting a threshold value; 
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encoding the bits among the candidate bits having rate-distortion slope values 
greater than threshold value; 

reducing the threshold value; and 

reconfiguring the candidate bits to include the most significant unencoded bits in 
5 each coefficient. 

5. A method according to claim 2 wherein the distortion reduction value for 
each bit is derived from a predicted rate-distortion slope. 

10 6. A method according to claim 5 wherein the predicted rate-distortion slope 

is generated according to the coding layer of each bit, a significance status of the 
coefficient indicating whether all previous bits in the coefficient are zeros, and the 
significance statuses of surrounding coefficients. 

15 7. A method according to claim 5 wherein the predicted rate distortion slope is 

derived for each bit according to a significance identification mode or a refinement mode. 

8. A method according to claim 7 wherein the significance identification mode 
encodes all significant bits in the coefficients defined as all bits up to a first binary 1 value 

20 and the refinement mode encodes all refinement bits defined as all bits after the first 
binary 1 value. 

9. A method according to claim 7 wherein the significance identification mode 
uses a QM-coder that estimates a probability that the bit is significant according to a 

25 significant status of the same and adjacent coefficients. 

10. A method according to claim 8 wherein the rate distortion slope is derived 
for the significant bits as follows: 

E[ADj] _ 2 - 25T ni 2 2 
^i>sig " EfARj]" l+H( Pi )/ Pi " f t&0 T nj > 

30 where the function f s (p) is the significance R-D slope modification factor defined as: 
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2.25 

f S (P) : 



H(p)' 
P 



T is a quantization step size determined by the coding layer nj, pj is probability of 
significance that a candidate bit b n .=l, and H(p) is entropy of a binary symbol. 



11. A method according to claim 8 wherein the rate distortion slope derived for 
the refinement bits is the following: 

E[ADj] 2 
Href" EfARi] -0 * 25 T q ' 

where T is a quantization step size determined by the coding layer nj. 



12. A method according to claim 8 wherein the rate distortion slope for the 
significance bits and the refinement bits are derived according to a Laplacian probability 
distribution. 



13. A method according to claim 7 including precomputing and storing a table 
containing the rate-distortion slope values for significance bit coding and refinement bit 
coding and indexing the rate-distortion slope values according to the coding layer and a 
coder state corresponding the probability of significance or insignificance that the most 
probable bit during the significant identification mode is a 1 or a 0. 

14. An embedded encoder for optimizing rate-distortion coding of a digitized 
image, comprising: 

a transformer encoding the digitized image forming a set of coefficients each 
comprising multiple bits; 

a rate-distortion slope calculator repeatedly analyzing candidate bits for each 
coefficient comprising a most significant unencoded bit from each coefficient and 
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determining individual distortion reduction values for individual bits representing a ratio 
of information content in the bit per cost in transmitting the bit; and 

a symbol selector selecting an order for encoding the bits according to the 
associated distortion reduction values. 

15. A system according to claim 14 wherein the rate-distortion slope calculator 
operates in a significance identification mode when all previously coded bits in the 
coefficient are zero and a refinement mode after a first nonzero bit is identified in the 
coefficient during the significance identification mode. 

16. A system according to claim 15 wherein the rate-distortion slope calculator 

comprises a lookup table including one refinement mode rate-distortion entry for each 

layer of coefficient bits and two significance identification mode rate-distortion entries, a 

first entry for a symbol 1 to be the most probable symbol, and 

a second entry for a symbol 0 to be the most probable symbol for each one of multiple 
QM-coder states each associated with a probability of significance. 
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